12 research outputs found
Why We Can Not Surpass Capacity: The Matching Condition
We show that iterative coding systems can not surpass capacity using only
quantities which naturally appear in density evolution. Although the result in
itself is trivial, the method which we apply shows that in order to achieve
capacity the various components in an iterative coding system have to be
perfectly matched. This generalizes the perfect matching condition which was
previously known for the case of transmission over the binary erasure channel
to the general class of binary-input memoryless output-symmetric channels.
Potential applications of this perfect matching condition are the construction
of capacity-achieving degree distributions and the determination of the number
required iterations as a function of the multiplicative gap to capacity.Comment: 10 pages, 27 ps figures. Forty-third Allerton Conference on
Communication, Control and Computing, invited pape
The Generalized Area Theorem and Some of its Consequences
There is a fundamental relationship between belief propagation and maximum a
posteriori decoding. The case of transmission over the binary erasure channel
was investigated in detail in a companion paper. This paper investigates the
extension to general memoryless channels (paying special attention to the
binary case). An area theorem for transmission over general memoryless channels
is introduced and some of its many consequences are discussed. We show that
this area theorem gives rise to an upper-bound on the maximum a posteriori
threshold for sparse graph codes. In situations where this bound is tight, the
extrinsic soft bit estimates delivered by the belief propagation decoder
coincide with the correct a posteriori probabilities above the maximum a
posteriori threshold. More generally, it is conjectured that the fundamental
relationship between the maximum a posteriori and the belief propagation
decoder which was observed for transmission over the binary erasure channel
carries over to the general case. We finally demonstrate that in order for the
design rate of an ensemble to approach the capacity under belief propagation
decoding the component codes have to be perfectly matched, a statement which is
well known for the special case of transmission over the binary erasure
channel.Comment: 27 pages, 46 ps figure
Conservation laws for coding
This work deals with coding systems based on sparse graph codes. The key issue we address is the relationship between iterative (in particular belief propagation) and maximum a posteriori decoding. We show that between the two there is a fundamental connection, which is reminiscent of the Maxwell construction in thermodynamics. The main objects we consider are EXIT-like functions. EXIT functions were originally introduced as handy tools for the design of iterative coding systems. It gradually became clear that EXIT functions possess several fundamental properties. Many of these properties, however, apply only to the erasure case. This motivates us to introduce GEXIT functions that coincide with EXIT functions over the erasure channel. In many aspects, GEXIT functions over general memoryless output-symmetric channels play the same role as EXIT functions do over the erasure channel. In particular, GEXIT functions are characterized by the general area theorem. As a first consequence, we demonstrate that in order for the rate of an ensemble of codes to approach the capacity under belief propagation decoding, the GEXIT functions of the component codes have to be matched perfectly. This statement was previously known as the matching condition for the erasure case. We then use these GEXIT functions to show that in the limit of large blocklengths a fundamental connection appears between belief propagation and maximum a posteriori decoding. A decoding algorithm, which we call Maxwell decoder, provides an operational interpretation of this relationship for the erasure case. Both the algorithm and the analysis of the decoder are the translation of the Maxwell construction from statistical mechanics to the context of probabilistic decoding. We take the first steps to extend this construction to general memoryless output-symmetric channels. More exactly, a general upper bound on the maximum a posteriori threshold for sparse graph codes is given. It is conjectured that the fundamental connection between belief propagation and maximum a posteriori decoding carries over to the general case